Light field display control method and apparatus, and light field display device

ABSTRACT

Embodiments of the present application disclose a light field display control method and apparatus and a light field display device. The light field display control method comprises: determining a partial depth distribution sub-region of content according to at least depth distribution information of the content; and tilting a first lenslet with respect to an original plane of a lenslet array of a light field display device according to at least a display depth of field (DoF) range of the light field display device and the depth distribution sub-region, wherein the first lenslet is a lenslet that is in the lenslet array and affects display of a first object, and the first object is a part, which is located in the depth distribution sub-region, of the content. The present application can improve display quality of an object, which is located in a partial depth distribution sub-region, of content to be displayed or content being displayed.

CROSS REFERENCE TO RELATED APPLICATION

The present application is based on and claims the benefit of priorityto Chinese Application No. 201511031335.0, filed on Dec. 31, 2015, theentire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present application relates to the field of light field displaytechnologies, and in particular, to a light field display control methodand apparatus, and a light field display device.

BACKGROUND

A light field display technology was proposed early in the 20^(th)century, and two representative implementation manners of the lightfield display technology are: a lenslet array-based implementationmanner proposed by Lippmann in 1908 and a parallax barriers-basedimplementation manner proposed by Ives in 1903. In recent years, asconsumer electronics impose diversified requirements on display devices,the light field display technology is applied to different scenarios anddevices, such as a 3D monitor, a wearable device, and a light fielddisplay device for vision correction. At present, the computing powerand display resolution of electronic devices gradually match with ahardware resource requirement of the light field display technology,which provides desirable support for the promotion and application ofthe light field display technology.

By means of a hardware structure similar to that of a conventional lightfield display technology, the light field display technology canimplement relatively flexible display effects, such as light field 3Ddisplay, light field projection display, light field near-to-eye displayon a wearable device, and vision correction by means of light fielddisplay.

SUMMARY

The following provides a brief summary about the present application, soas to provide basic comprehension about some aspects of the presentapplication. It should be understood that, the summary is not anexhaustive summary of the present application. It is neither intended todetermine the key part or important part of the present application norintended to limit the scope of the present application. The objectivethereof is merely to provide some concepts in a simplified form, toserve as a prelude for subsequent more detailed descriptions.

The present application provides a light field display control methodand apparatus and a light field display device.

According to a first aspect, an embodiment of the present applicationprovides a light field display control method, comprising:

determining a partial depth distribution sub-region of content accordingto at least depth distribution information of the content; and

tilting a first lenslet with respect to an original plane of a lensletarray of a light field display device according to at least a displaydepth of field (DoF) range of the light field display device and thedepth distribution sub-region, wherein the first lenslet is a lensletthat is in the lenslet array and affects display of a first object, andthe first object is a part, which is located in the depth distributionsub-region, of the content.

According to a second aspect, an embodiment of the present applicationfurther provides a light field display control apparatus, comprising:

a depth distribution sub-region determining module, configured todetermine a partial depth distribution sub-region of content accordingto at least depth distribution information of the content; and

a tilt control module, configured to tilt a first lenslet with respectto an original plane of a lenslet array of a light field display deviceaccording to at least a display DoF range of a light field displaydevice and the depth distribution sub-region, wherein the first lensletis a lenslet that is in the lenslet array and affects display of a firstobject, and the first object is a part, which is located in the depthdistribution sub-region, of the content.

According to a third aspect, an embodiment of the present applicationprovides another light field display control apparatus, comprising:

a processor, a communications interface, a memory, and a communicationsbus, wherein the processor, the communications interface, and the memorycommunicate with each other by means of the communications bus;

the memory is configured to store at least one instruction, and theinstruction causes the processor to perform the following operations:

determining a partial depth distribution sub-region of content accordingto at least depth distribution information of the content; and

tilting a first lenslet with respect to an original plane of a lensletarray of a light field display device according to at least a displayDoF range of the light field display device and the depth distributionsub-region, wherein the first lenslet is a lenslet that is in thelenslet array and affects display of a first object, and the firstobject is a part, which is located in the depth distribution sub-region,of the content.

According to a fourth aspect, an embodiment of the present applicationprovides a light field display device, comprising:

a monitor;

a lenslet array, wherein the lenslet array comprises multiple tiltablelenslets distributed in the array; and

any light field display control apparatus described above, wherein thelight field display control apparatus is connected to the monitor andthe lenslet array.

In the technical solutions provided by the embodiments of the presentapplication, the first lenslet is tilted with respect to the originalplane of the lenslet array according to at least a current display DoFrange of the light field display device and a depth distributionsub-region on which image quality control needs to be performed, tochange a display image distance of the first object in space by means ofcontrol over the tilt of the first lenslet, so that an average displayimage distance of a virtual image which is formed after the first objectis displayed by the first lenslet is distributed in the current displayDoF range of the light field display device as far as possible, therebyimproving display quality of the first object displayed by the firstlenslet. This solution can avoid complex computation on source contentof a light field, and display quality of a same object can be adjustedby tilting a corresponding lenslet with respect to the original plane ofthe lenslet array. The solution is simple and easy to control.

Through the following detailed description of optional embodiments ofthe present application with reference to the accompanying drawings,these and other advantages of the present application will be clearer.

BRIEF DESCRIPTION OF THE DRAWINGS

The present application may be better comprehended with reference to thefollowing description provided in combination with the accompanyingdrawings, wherein same or similar reference numerals are used in all theaccompanying drawings to represent same or similar components. Theaccompanying drawings together with the following detailed description,which are incorporated in the specification and form a part of thespecification, are used to further illustrate the optional embodimentsof the present application and explain the principle and advantage ofthe present application. In the accompanying drawings:

FIG. 1 is a flowchart of a light field display control method accordingto an embodiment of the present application;

FIG. 2a illustrates a first example of a light field display device andlight field display thereof according to an embodiment of the presentapplication;

FIG. 2b illustrates a second example of a light field display device andlight field display thereof according to an embodiment of the presentapplication;

FIG. 3 illustrates an example of determining a first lenslet accordingto an embodiment of the present application;

FIG. 4 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application;

FIG. 5 is a logic block diagram of a depth distribution sub-regiondetermining module according to an embodiment of the presentapplication;

FIG. 6 is a logic block diagram of a tilt control module according to anembodiment of the present application;

FIG. 7 is a logic block diagram of an expected display depth informationacquiring sub-module according to an embodiment of the presentapplication;

FIG. 8 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application;

FIG. 9 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application; and

FIG. 10 is a schematic structural diagram of a light field displaydevice according to an embodiment of the present application.

A person skilled in the art should understand that, elements in theaccompanying drawings are merely shown for the purpose of simplicity andclarity, but are not necessarily drawn proportionally. For example,sizes of some elements in the accompanying drawings may be enlargedrelative to other elements, to help understand the embodiments of thepresent application.

DETAILED DESCRIPTION

Exemplary embodiments of the present application are described in detailin the following through the accompanying drawings. For clarity andsimplicity, not all features of actual implementation manners aredescribed in the specification. However, it should be understood that ina process of developing any such actual embodiment, many decisionsspecific to the implementation manner need to be made to implementspecific objectives of developers, for example, meeting restrictiveconditions related to a system and service, and the restrictiveconditions may vary according to different implementation manners.Moreover, it should be further understood that, although the developmentwork may be very complex and time consuming, the development work ismerely a routine task for a person skilled in the art who is benefitedfrom the content of the present disclosure.

Here, it should be further noted that, to prevent the presentapplication from being blurred by unnecessary details, merely apparatusstructures and/or processing steps closely related to the solutions ofthe present application are described in the accompanying drawings andthe specification, and expressions and descriptions about components andprocessing that are less related to the present application and known bya person of ordinary skill in the art are omitted.

Specific implementation manners of the present application are furtherdescribed in detail with reference to the accompanying drawings (samereference numerals in several accompanying drawings represent sameelements) and embodiments. The following embodiments are used todescribe the present application, but are not intended to limit thescope of the present application.

A person skilled in the art should understand that terms such as “first”and “second” in the present application are merely used to distinguishdifferent steps, devices, modules or the like, and the terms neitherrepresent any specific technical meanings nor indicate necessary logicorders between them.

FIG. 1 is a flowchart of a light field display control method accordingto an embodiment of the present application. The light field displaycontrol method provided in an embodiment of the present application maybe executed by a light field display control apparatus, and the lightfield display control apparatus may execute the light field displaycontrol method in application programs that comprise but not limited to:image presentation, video playback, and the like, to perform imagedisplay control. A device manifestation form of the light field displaycontrol apparatus is not limited. For example, the light field displaycontrol apparatus may be an independent component, and the component isin coordination and communication with a light field display device; orthe light field display control apparatus may be integrated in a lightfield display device as a functional module, and the light field displaydevice may comprise but not limited to: an electronic device having alight field display capability. For example, the light field displaydevice may comprise but not limited to: a near-to-eye light fielddisplay device, and the near-to-eye light field display device maycomprise but not limited to: a smart helmet, smart glasses, and thelike. Specifically, as shown in FIG. 1, a light field display controlmethod provided in an embodiment of the present application comprises:

S101: Determine a partial depth distribution sub-region of contentaccording to at least depth distribution information of the content.

The content is content being displayed or to be displayed. The depthdistribution information of the content means distribution informationabout various parts of the content in a depth direction when the contentis in a displayed state. The partial depth distribution sub-region maybe determined according to at least the depth distribution informationof the content, as a depth distribution sub-region on which displayquality control needs to be performed.

S102: Tilt a first lenslet with respect to an original plane of alenslet array of a light field display device according to at least adisplay DoF range of the light field display device and the depthdistribution sub-region, wherein the first lenslet is a lenslet that isin the lenslet array and affects display of a first object, and thefirst object is a part, which is located in the depth distributionsub-region, of the content.

The light field display device comprises a monitor and a lenslet arraythat are arranged close to each other, as shown in FIG. 2a . The monitormay be a continuous monitor or the monitor may be a screen formed bymultiple connected display units that are distributed in an array,wherein the monitor may be divided into multiple display sub-regionswhen displaying content, each display sub-region comprises multipledisplay pixels distributed in an array, each display sub-region candisplay a sub-image, and the sub-image is a part of an image of thecontent. The lenslet array may also be referred to as a micro-lensarray, and comprises multiple lenslets (or referred to as micro lenses)distributed in an array. A propagation direction of a light ray from themonitor is changed by at least one lenslet in the lenslet array, and thelight ray with the changed propagation direction forms an image on aretina of a user after passing through an eye of the user (similar to alens). If a partial image formed on the retina is a relatively smallaverage circle of confusion, it is equivalent to that the user can see aclear virtual image which is displayed in space and corresponds to thepartial image. On the contrary, if a partial image formed on the retinahas a relatively large average circle of confusion, it is equivalent tothat the user sees a vague virtual image which is displayed in space andcorresponds to the partial image. In addition, because the monitor andthe lenslet array are arranged close to each other (wherein it should benoted that, a relative distance between components in the figure merelyprovides an exemplary illustration, and does not represent an actualdistance between the components), the light ray whose propagationdirection has been changed by the at least one lenslet forms a virtualimage on a side, which is away from the user, of the monitor. Thisvirtual image corresponds to the image formed on the retina of the user,and is equivalent to an image seen by a human eye through the lensletarray; the virtual image may present a certain depth distribution inspace, and the distribution of a circle of confusion of the virtualimage in a current first display plane of the light field display devicecorresponds to the distribution of a circle of confusion of the imageformed on the retina. For example, if some part of the virtual image hasa relatively large average circle of confusion in the first displayplane, an image of this part formed on the retina also has a relativelylarge average circle of confusion, and vice versa.

In a light field display process, the content displayed by the lightfield display device has a display DoF range in space, wherein thedisplay DoF range is a range that has a particular depth width in adepth direction, and the range comprises a depth position of the firstdisplay plane; the first display plane is a display plane that iscorresponding to a focusing distance for viewing in the depth directionand perpendicular to the depth direction. The focusing distance forviewing may be a default human eye focusing distance (for example, it isconsidered by default that an optimal distance for viewing a virtualimage formed in the first display plane is 25 cm, and at this viewingdistance, a human eye can clearly see the virtual image formed in thefirst display plane; in other words, a clearly image of the contentdisplayed by the first display plane can be formed on the retina of thehuman eye located at the focusing distance for viewing). Alternatively,the focusing distance for viewing may be an actual human eye focusingdistance, or the like. In an actual application, after a suitablefocusing distance for viewing is determined, the first display planecorresponding to the focusing distance for viewing can be determined. Anear-to-eye light field display device is used as an example fordescription. When the near-to-eye light field display device is used(for example, wearing smart glasses having a light field displaycapability) to watch the displayed content, a distance L between thehuman eye and the near-to-eye light field display device is relativelyfixed. A difference value between the focusing distance for viewing andthe distance L is calculated, and a depth position corresponding to thedifference value is determined in the depth direction, wherein thisdepth position is the depth position of the first display plane. Afterthe first display plane is determined, the display DoF range of thelight field display device may be determined. For example, a permissiblecircle of confusion may be determined according to one or moreparameters of a display quality expectation, a human eye characteristic,and the like; average imaging circles of confusion at differentdistances from the first display plane in the depth direction arecalculated; and a continuous depth range, which comprises the depthposition of the first display plane and of which an average imagingcircle of confusion is less than the permissible circle of confusion, isdetermined as the display DoF range. Further, the display DoF range maybe optimized with reference to the visual sensitivity of a human eye tothe clearness of content displayed at different depths. For example, thehuman eye is more visually sensitive to the clearness of contentdisplayed at a near place than to the clearness of content displayed ata further place. Therefore, with reference to the depth position of thefirst display plane, a permissible circle of confusion at a depthposition close to the depth position of the first display plane may bedetermined as a lower permissible circle of confusion threshold (C1),and a permissible circle of confusion at a depth position away from thedepth position of the first display plane may be determined as an upperpermissible circle of confusion threshold (C2), thereby determining thedisplay DoF range. The determined display DoF range is expressed as aparticular depth range that comprises the depth position of the firstdisplay plane and that is asymmetrically distributed along the depthposition of the first display plane. Certainly, the foregoing method fordetermining the first display plane and the display DoF range is merelyan illustrative description, and a person skilled in the art can alsouse other determining methods, which are not limited in the embodimentsof the present application. A virtual image displayed in the display DoFrange is very clearly for a human eye at the focusing distance forviewing corresponding to the first display plane, while a virtual imagedisplayed outside the display DoF range is very vague for a for a humaneye at the focusing distance for viewing corresponding to the firstdisplay plane.

During implementation of the embodiments of the present application, theinventor of the present application finds that, if the clearness ofdifferent parts of the content needs to be adjusted, the current firstdisplay plane of the light field display device may be adjusted, andafter the first display plane is changed, it is equivalent to that thedisplay DoF range of the light field display device is redetermined. Apart clearly displayed by the light field display device can be adjustedbased on the depth distribution information of the displayed content andthe re-determined display DoF range. For example, content that isclearly displayed by the light field display device currently is aforeground part having a relatively small depth, and by adjusting thefirst display plane, the content clearly displayed content can bechanged to be a background part having a relatively large depth, or thelike. To re-determine the first display plane, source content displayedby the light field display device needs to be processed. This requires alarge amount of computation, the computation is highly complex, and thedisplay speed is low. Therefore, the present application provides a newsolution.

The inventor notices that by tilting a lenslet, a display image distanceof a first object can be changed. Therefore, in the technical solutionprovided by the embodiments of the present application, this displayimage distance adjustment means is fully utilized: a first lenslet istilted with respect to an original plane of the lenslet array (whereinthe original plane is a plane in which the lenslet array is located whenall lenslets of the lenslet array are in a non-tilted state) accordingto a current display DoF range of the light field display device and adepth distribution sub-region on which image quality control needs to beperformed, to change a display image distance of the first object inspace at least by means of control over the tilt of the first lenslet,so that an average display image distance of a virtual image which isformed after the first object is displayed by the first lenslet isdistributed in the current display DoF range of the light field displaydevice as far as possible, thereby improving display quality of thefirst object displayed by the first lenslet. A near-to-eye light fielddisplay is used as an example for description. Assuming that a distancebetween the first display plane and a human eye is 25 cm (that is, thefocusing distance for viewing is 25 cm), an image formed in a particulardepth range of the first display plane (that is, the display DoF range),for example, the depth range of [25 cm−5 cm, 25 cm+7 cm], is clear tothe human eye, and if a user wants to see an image outside the displayDoF range, for example, if the user wants to clearly see an image near aposition which is 35 cm away from the human eye, a corresponding lensletmay be tilted with respect to the original plane of the lenslet array,so that images of a same object are displayed in the display DoF range,thereby improving the imaging clearness of the object, to allow the userto see the object clearly. This solution can avoid complex computationon source content of a light field, and display quality of a same objectcan be adjusted by tilting a corresponding lenslet with respect to theoriginal plane of the lenslet array. The solution is simple and easy tocontrol.

The first lenslet is a lenslet that is in the lenslet array of the lightfield display device and affects display of the first object. The firstobject is a part, which is in the depth distribution sub-region, of thecontent, and if the content is regarded as a complete display image, thefirst object is a part of the complete display image. Display of thefirst object may be affected by one or more lenslets. Therefore, in anactual application, there may be one or more first lenslets. Tiltcontrol may be performed on the one or more first lenslets, so as toimprove the clearness of the displayed first object and improve displayquality.

When the first object is displayed by using, for example, the lightfield display device, affected by factors such as the property of anoptical wave and an aberration, imaging light rays of points on theobject usually cannot be converged at one point, but form a diffusionprojection in a shape of a circle or an ellipse or in anothercircle-like shape, that is, a circle of confusion, which is also knownas disk of confusion, circle of indistinctness, blur circle, or blurspot. The size of a circle of confusion for imaging of a point may becalculated according to the following formula:

$\begin{matrix}{c_{a} = {\frac{{d_{a} - d_{0}}}{d_{a}}E}} & (1)\end{matrix}$

In the formula above: E represents a size of a human eye pupil, forexample, a diameter of a human eye pupil; d_(a) represents a focusingdistance for viewing, that is, a distance between a current firstdisplay plane of a light field display device and the human eye pupil;d₀ represents a display depth of an object, for example, an averagedistance, in a depth direction, between a virtual image of the objectand a human eye located at the focusing distance for viewing; c_(a)represents a size of a circle of confusion (for example, a diameter of acircle of confusion) generated during imaging, at the human eye, of animage point having a display depth of d₀.

A circle of confusion distribution, which is in the first display planeof the light field display device, of a virtual image that is formed inspace after the first object is displayed by the tilted first lensletcorresponds to a circle of confusion distribution, which is on theretina, of an image that is formed on the retina after the virtual imageundergoes an equivalent lens effect of the human eye. Usually, if acircle of confusion has a relatively small size, an image of an objectcorresponding to the circle of confusion is relatively clear, and can beregarded as an in-focus image of the point with respect to the humaneye, wherein the image is clear; correspondingly, if a size of a circleof confusion exceeds a particular permissible range, an image of anobject corresponding to the circle of confusion is relatively blur. Inthe embodiments of the present application, the tilt of the firstlenslet is controlled, and a convergence condition for tilt control ofthe first lenslet is as follows: an average circle of confusion of animage, which is formed on the first display plane of the light fielddisplay device or on the retina after a corresponding object isdisplayed by the tilted lenslet, is less than or equal to a permissiblecircle of confusion. In other words, by means of control over the tiltof the lenslet, an average circle of confusion of images, which areformed on the first display plane of the light field display device oron the retina by points of the first object displayed by the tiltedfirst lenslet, is less than or equal to a permissible circle ofconfusion. As shown in FIG. 2b , an average circle of confusion of aretina image, which corresponds to a virtual image of the first objectdisplayed by the tilted first lenslet, is reduced, and therefore averageclearness of points in the first object is improved. It should be notedthat, the term “points in the first object” expresses a relativeconcept, for example, a part displayed by an individual display pixel inthe monitor may be used as a point in the first object, which is notlimited in the embodiments of the present application.

In any technical solution provided in the embodiments of the presentapplication, a manner for determining the depth distribution sub-regionon which display quality control needs to be performed in the content isvery flexible.

For example, the depth distribution information of the content andinterest information may be combined to determine the depth distributionsub-region, that is, region of interest (ROI) information and the depthdistribution information may be acquired, and the depth distributionsub-region may be determined according to the ROI information and thedepth distribution information. The manner for acquiring the ROIinformation may be determined according to an actual requirement, and isvery flexible. For example, degree of interest indication information ofthe content may be acquired, and the ROI information is determinedaccording to the degree of interest indication information. The degreeof interest indication information may be acquired from a user or adevice, and used as a basis for determining the ROI information of thecontent. Content and an expression manner of the degree of interestindication information may be determined according to an actualrequirement, and the degree of interest is used to represent a relativevalue indicating how much the user or device is interested in differentdepth sub-regions of the content; the user or device usually has a highdegree of interest in an ROI of the content, and usually has a lowdegree of interest in a non-ROI. The ROI may comprise but not limited toone or more of the following types: at least one area of the contentthat the user selects (that is, a user selected region of the content),at least one area of the content that the user gazes at (that is, a usergaze region of the content), and the like. In this solution, the depthdistribution information of the content and the interest information arecombined to determine a partial depth distribution sub-region of thecontent as the depth distribution sub-region on which display qualitycontrol needs to be performed, so that the determining of the depthdistribution sub-region fits an actual requirement of the user moredesirably, and a personalized application requirement of the user can bebetter satisfied.

For another example, the depth distribution information of the contentand image analysis on the content may be combined to determine the depthdistribution sub-region, that is, image analysis may be performed on thecontent, and the depth distribution sub-region is determined accordingto a result of the image analysis and the depth distributioninformation. For example: human body recognition is performed on thecontent, a relatively high degree of interest is assigned to a depthdistribution sub-region, which corresponds to a human body, of thecontent according to a result of the recognition, a relatively lowdegree of interest is assigned to areas of other depths in the content,and the like. In this solution, the result of the image analysis on thecontent and the depth distribution information of the content arecombined to automatically determine the depth distribution sub-region onwhich display quality control needs to be performed, so that thedetermining of the depth distribution sub-region is more intelligent,and the efficiency and universality of determining the depthdistribution sub-region are improved.

After the depth distribution sub-region is determined, the first lensletmay be tilted with respect to the original plane of the lenslet array ofthe light field display device according to at least the display DoFrange of the light field display device and the depth distributionsub-region. A specific implementation manner of tilt control is veryflexible.

In an optional implementation manner, the tilting a first lenslet withrespect to an original plane of a lenslet array of a light field displaydevice according to at least a display DoF range of the light fielddisplay device and the depth distribution sub-region comprises:determining expected display depth information of the first objectaccording to the display DoF range and the depth distributionsub-region; determining an expected tilt angle of the first lensletaccording to at least a focal length of the first lenslet and theexpected display depth information; and tilting the first lenslet withrespect to the original plane of the lenslet array according to at leastthe expected tilt angle. After the first lenslet is tilted with respectto the original plane of the lenslet array, as shown in FIG. 2b , theplane in which the first lenslet is located is no longer parallel to theplane in which the monitor is located. With reference to the Scheimpflugprinciple, when an extended plane of a plane where an image, which isformed by means of the first lenslet, of the first object in the depthdistribution sub-region is located (the plane is referred to as anobject display plane below), an extended plane of a plane in which thetilted first lenslet is located, and an extended plane of a plane inwhich the monitor is located intersect at a straight line (which may bereferred to as a Scheimpflug line), the first object in the depthdistribution sub-region can obtain a maximum clear imaging area and aminimum average circle of confusion; a tilt angle of the first lensletsatisfying the Scheimpflug principle is the expected tilt angle in theembodiment of the present application.

The object display plane may be obtained by means of fitting accordingto an expected distribution, in the display DoF range, of each imagepoint in the depth distribution sub-region. For example, an optionalimplementation manner is:

(a1) Assuming that the content is a clear 2D image (wherein after lightfield display, the 2D image is displayed as a 3D image having aparticular display depth distribution in space), an image coordinate set{(x_(i),y_(i))} that comprises respective coordinates, in the 2D image,of points in the first object in a partial depth distribution sub-regionof the 2D image may be determined according to depth distributioninformation of the 2D image.

(b1) According to the determined image coordinate set of the points, afirst space coordinate set of respective virtual image pointscorresponding to the points is determined, and first space coordinatesof any image point may be expressed as p(x_(i)′,y_(i)′,d_(i)′) whereinx_(i)′=M₁x_(i), y_(i)′=M₂y_(i), M is a known scaling coefficient, forexample, M1 is a known X-direction scaling coefficient and M2 is a knownY-direction scaling coefficient, and d_(i)′ represents an image distanceof the image point, that is, an image distance before the image point ismapped.

(c1) According to the determined first space coordinate set and thecurrent display DoF range of the light field display device, coordinatesof each point in the first space coordinate set are mapped into aparticular depth sub-range of the display DoF range respectively,thereby obtaining a second space coordinate set, and second spacecoordinates of any image point may be expressed asp′(x_(i)″,y_(i)″,d_(i)″), wherein x_(i)″ represents a horizontalcoordinate of the image point in the first display plane; y_(i)″represents a vertical coordinate of the image point in the first displayplane; and d_(i)″ at represents an image distance of the image pointafter mapping. The mapping may be performed according to a particularrule. For example, on the premise of maintaining a relative depthrelationship of image points in the depth distribution sub-region, theimage points are mapped to a particular depth sub-range in the displayDoF range. In other words, before adjustment, if a relative depthrelationship between any two image points: image point p₁ and imagepoint p₂, is that image distances of the two image points satisfy thatd_(p2)≥d_(p1), then the image distances of the two image points afterthe adjustment satisfy that d_(p2)′≥d_(p1)′. What is changed before andafter the adjustment is a difference value between image distances. Forexample, a difference (d_(p2)′−d_(p1)′) between the image distances ofthe two image points before the adjustment is greater than a difference(d_(p2)′−d_(p1)′) between the image distances of the two image pointsafter the adjustment, that is, (d_(p2)−d_(p1))≥(d_(p2)′-d_(p1)′). Thisis equivalent to a certain degree of depth compression on the firstobject in the depth distribution sub-region, so as to improve displayquality of more parts of the first object. Optionally, proportionalmapping may be performed on the premise of keeping the relative depthrelationship of image points unchanged. For example, if four imagepoints whose image distances are {25, 28, 42, 45} respectively need tobe mapped into a display depth sub-region [30, 40], image distances ofthe image points after the mapping are {25+(30−25)*25/25,(28+30−25)*25/28, (42−5)*45/42, (45−5)*45/45} respectively, and in thisway, the relative depth relationship of the image points is keptunchanged before and after the mapping.

(d1) A plane is obtained by means of fitting according to the secondspace coordinate set. This step is equivalent to providing a point cloudand obtaining a plane by means of fitting according to the point cloud,to make a sum of distances from points in the point cloud to the planeminimum. Further, during the fitting process, it can be ensured that theminimum distance sum is less than a preset value; otherwise, multipleplanes may be obtained by means of fitting until a minimum distance sumis less than the preset value. An algorithm used in plane fittingbelongs to the prior art. For example, a Random Sample Consensus (RANSC)is used, but the algorithm is not limited thereto, and details are notdescribed herein.

Further optionally, the expected tilt angle of the first lenslet may becalculated according to the following formula:

$\begin{matrix}{{\tan\; Ø} = {\frac{v}{{v\;\cos\;\theta} - f}\sin\;\theta}} & (3)\end{matrix}$

In the foregoing formula: θ represents a tilt angle of the first lensletwith respect to the original plane of the lenslet array, that is, anexpected tilt angle of the first lenslet; f represents a focal length ofthe first lenslet, Ø represents an angle between the object imagingplane and the plane in which the monitor is located, and v representsexpected display depth information of the first object. The firstlenslet is tilted with respect to the original plane of the lensletarray according to at least the expected tilt angle determined in thissolution, so that a tilt angle of the first lenslet with respect to theoriginal plane of the lenslet array (for example, an angle between anoptical axis of the tilted first lenslet and the original plane of thelenslet array) is close to the expected tilt angle as much as possibleor even equal to the expected tilt angle. This solution improves tiltcontrol efficiency of the first lenslet, and the first object displayedby the tilted first lenslet can obtain a relatively large clear imagingarea.

In an actual application, the tilt of the first lenslet with respect tothe original plane of the lenslet array may be controlled according tothe expected tilt angle, so that the tilt angle of the first lensletwith respect to the original plane of the lenslet array is close to theexpected tilt angle as much as possible or even equal to the expectedtilt angle.

Alternatively, a tilt direction of the first lenslet may also bedetermined according to a relative distribution of the depthdistribution sub-region and the display DoF range in the depthdirection, and the first lenslet is tilted with respect to the originalplane of the lenslet array according to the tilt direction and theexpected tilt angle. By means of this solution, tilt control efficiencyof the first lenslet can be improved, so that tilt control on the firstlenslet satisfies a convergence condition of the average circle ofconfusion as quickly as possible. Specifically speaking, the tiltedfirst lenslet can change the display image distance of the first object,and if an actual image distance of the first object needs to be adjustedso that it falls in the display DoF range of the light field displaydevice, the tilt direction of the first lenslet may be determinedaccording to the relative distribution of the depth distributionsub-region and the display DoF range in the depth direction, to improvethe tilt control efficiency of the first lenslet. For example, as shownin FIG. 2b , if the depth distribution sub-region of the first object islocated at a relatively deep position with respect to the display DoFrange (for example, an image distance d_(p2) of an image point p₂obtained after a point is displayed by the non-tilted first lenslet islocated at a relatively deep position with respect to the display DoFrange), it is expected to adjust the display image distance of the firstobject to a depth position with a relatively small depth value (forexample, it is expected that the image point p₂, which has the imagedistance d_(p2) when the point is displayed by the non-tilted firstlenslet, forms an image at an image distance d_(p2)″ by means of thetilted first lenslet, and d_(p2)>d_(p2)″), and in this case, it may bedetermined that the tilt direction of the first lenslet is a firstdirection, wherein the first direction is a direction of increasing anangle between the plane in which the first lenslet is located and theplane in which the monitor is located. The first lenslet is tiltedtowards the first direction with respect to the original plane of thelenslet array by the expected tilt angle, and in this way, theconvergence condition of the average circle of confusion can besatisfied as quickly as possible in the tilt control process of thefirst lenslet, thereby improving the tilt control efficiency of thefirst lenslet. For another example, if the depth distribution sub-regionof the first object is located at a relatively shallow position withrespect to the display DoF range, it is expected to adjust the displayimage distance of the first object to a depth position with a relativelylarge depth value, and in this case, it may be determined that the tiltdirection of the first lenslet is a second direction, wherein the seconddirection is a direction of decreasing an angle between the plane inwhich the first lenslet is located and the plane in which the monitor islocated. The first lenslet is tilted towards the second direction withrespect to the original plane of the lenslet array by the expected tiltangle, and in this way, the convergence condition of the average circleof confusion can be satisfied as quickly as possible in the tilt controlprocess of the first lenslet, thereby improving the tilt controlefficiency of the first lenslet.

In another optional implementation manner, the tilting a first lensletwith respect to an original plane of a lenslet array of a light fielddisplay device according to at least a display DoF range of the lightfield display device and the depth distribution sub-region comprises:determining expected display depth information of the first objectaccording to the display DoF range and the depth distributionsub-region; determining a permissible tilt angle range of the lensletaccording to at least a focal length of the first lenslet and theexpected display depth information; and tilting the first lenslet withrespect to the original plane of the lenslet array according to at leastthe permissible tilt angle range. By means of this solution, the firstlenslet is tilted with respect to the original plane of the lensletarray, and an angle between the tilted first lenslet and the originalplane of the lenslet array (for example, an angle between an opticalaxis of the tilted first lenslet and the original plane of the lensletarray) falls in the permissible angle range. The permissible angle rangemay be flexibly determined according to a requirement that the tiltedlenslet can reduce an average circle of confusion in imaging of acorresponding object, which is not limited in the embodiment of thepresent application. This solution improves the tilt control efficiencyof the first lenslet, and can improve the clearness of the first objectdisplayed by the tilted first lenslet.

The permissible angle range may be predetermined according to therequirement that the tilted lenslet can reduce the corresponding averagecircle of confusion, and a determining method may comprise but notlimited to: determining by means of experiment, determining by means ofanalog simulation, determining by means of formula derivation, and thelike. Optionally, the determining a permissible tilt angle range of thefirst lenslet according to at least the expected display depthinformation comprises: determining an expected tilt angle of the firstlenslet according to at least a focal length of the first lenslet andthe expected display depth information; and determining a permissibleangle range according to at least the expected tilt angle. For a methodfor determining the expected tilt angle, refer to the description above.After the expected tilt angle is determined, with reference to factorssuch as a tilt adjustment precision limitation of the lenslet and adisplay quality requirement on an object, an angle range of the expectedtilt angle within a certain permissible error range is used as thepermissible angle range. The permissible angle range determined by usingthis solution is relatively reasonable, and control over the tilt of thelenslet with reference to the imaging plane performed based on thepermissible angle range is efficient and easy to implement.

In an actual application, the tilt of the first lenslet with respect tothe original plane of the lenslet array may be controlled according tothe permissible angle range, so that the tilt angle of the first lensletwith respect to the original plane of the lenslet array falls in thepermissible angle range.

Alternatively, a tilt direction of the first lenslet may be determinedaccording to a relative distribution of the depth distributionsub-region and the display DoF range in the depth direction; and thefirst lenslet is tilted with respect to the original plane of thelenslet array according to the tilt direction and the permissible tiltangle range. For a method for determining the tilt direction of thefirst lenslet, refer to the description above. By means of thissolution, the tilt control efficiency of the first lenslet can beimproved, so that tilt control on the first lenslet satisfies aconvergence condition of the average circle of confusion as quickly aspossible.

Further, in any technical solution provided by the embodiments of thepresent application, the method for determining the expected displaydepth information is very flexible, and is not limited in theembodiments of the present application.

Optionally, the determining expected display depth information of thefirst object according to at least the display DoF range comprises:determining any display depth in the display DoF range as the expecteddisplay depth information of the first object. For example, the displayDoF range of the light field display device is [V_(close),V_(far)], anda display depth in the display DoF range may be determined as theexpected display depth information of the first object, that is,expected display depth information V′∈[V_(close),V_(far)] of each pointin the first object. In an actual application, a preferable displaydepth in the display DoF range may be determined as the expected displaydepth information of the first object according to one or more factorssuch as a focal length adjustment capability of the first lenslet and arelative distribution of objects in different depth distributionsub-regions of the content, and the expected display depth informationis used as a basis for tilt control of the first lenslet, therebyimproving the flexibility of solution implementation.

Optionally, the determining expected display depth information of thefirst object according to at least the display DoF range comprises:determining a display depth, which is in the display DoF range and closeto the depth distribution sub-region, as the expected display depthinformation of the first object according to a relative distribution ofthe depth distribution sub-region and the display DoF range in the depthdirection. The term “close to” is a relative concept taking apermissible range into consideration, and in an actual application, theexpected display depth information may be determined with reference to adepth position V_(focus) of a current display focus plane of the lightfield display device. For example, if the depth distribution sub-regioncorresponding to the first object is located at a relatively deep areawith respect to the display DoF range, a display depth in the range of[V_(focus),V_(far)] (for example, a depth position as close to V_(far)as possible in the range of [V_(focus),V_(far)]) may be determined asthe expected display depth information of the first object; if the depthdistribution sub-region corresponding to the first object is located ata relatively shallow area with respect to the display DoF range, adisplay depth in the range of [V_(focus),V_(far)] (for example, a depthposition as close to V_(close) as possible in the range of[V_(focus),V_(far)]) may be determined as the expected display depthinformation of the first object. By means of such processing, a relativerelationship of display depths between objects, of the content,distributed at different depths can be kept unchanged as far aspossible. An object having a greater depth has a greater display imagedistance than an object having a smaller depth, so that the sense ofdepth when the user watches the image is not reduced while the clearnessof the image is improved, thereby improving user experience.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, before the tilting a first lensletwith respect to an original plane of a lenslet array of a light fielddisplay device, the method further comprises: determining a lenslet,which is in the lenslet array and affects display of the first object,as the first lenslet. By means of this solution, one or more lenslets,which are in the lenslet array of the light field monitor and affectdisplay of the first object, may be determined as the first lenslets,and then tilt control is performed on the one or more first lenslets, tochange the display image distance of the first object displayed by thecorresponding first lenslet, so that the display image distance falls inthe display DoF range of the light field display device, therebyimproving the display quality of the first object.

Further optionally, the determining a lenslet, which is in the lensletarray and affects display of the first object, as the first lensletcomprises: determining a first display sub-region according to theexpected display depth information, the focusing distance for viewing,and a pupil size of a human eye, wherein the first display sub-region isa display sub-region that is in the monitor of the light field displaydevice and affects display of the first object; and determining,according to a mapping relationship between display sub-regions in themonitor and lenslets in the lenslet array, a lenslet which is in thelenslet array and corresponding to the first display sub-region as thefirst lenslet. In some cases of light field display, content displayedon the monitor and a virtual image that a user sees through a lensletarray are the same in terms of content but have different imageexpression forms. The virtual image that the user sees through thelenslet array is usually consistent with an image seen by a human eye ina natural environment, while the monitor displays multiple sub-images,each sub-image corresponds to a local part of the virtual image, andobjects represented by different sub-images may be partially the same.For example, light field display is performed by using triple angularsamples (that is, 3×3 display sub-regions are used to represent displayinformation of a same object from different angles). For example, a sameobject such as a beak may be shown in 9 sub-images, lensletscorresponding to the display sub-regions that display the 9 sub-imagesmay be determined as the first lenslets to be adjusted, and the focallengths of the first lenslets are adjusted to improve the imageclearness of the object. In an actual application, the first displaysub-region may be determined according to at least the depthdistribution information of the content, and then the first lenslet isfurther determined according to the first display sub-region. Forexample, referring to FIG. 3, determining a first lenslet that affectsimaging of a point of the first object is used as an example to describethe method for determining the first lenslet:

(a2) Assuming that the content is a clear 2D image (wherein after lightfield display, the 2D image is displayed as a 3D image having aparticular display depth distribution in space), image coordinates{(x_(i), y_(i))}, in the 2D image, of points in the first object in apartial depth distribution sub-region of the 2D image may be determinedaccording to depth distribution information of the 2D image. It shouldbe noted that, the determined image coordinates of the points may be ina concentrated distribution or in a dispersed distribution. A point inthe first object is used as an example for description.

(b2) According to the determined image coordinates of the point, spacecoordinates p(x_(i)′,y_(i)′) of a virtual image point P corresponding tothe point are determined, wherein x_(i)′=M₁x_(i), y_(i)′=M₂y_(i), M is aknown scaling coefficient, for example, M1 is a known X-directionscaling coefficient and M2 is a known Y-direction scaling coefficient.

(c2) The virtual image point P may be regarded as a virtual lightsource. Entering of a light ray emitted by the virtual light source intoa human eye is related to a distance between the virtual light sourceand the human eye and the pupil size of the human eye, while thedistance between the virtual light source and the human eye isequivalent to a display depth of the virtual image point P, and adiameter of the pupil of the human eye can be acquired in advanceaccording to an empirical value, an experiment value, or a detectedvalue. After the two pieces of information are determined, a light conewith the virtual image point P as a vertex is also determinedcorrespondingly, and light rays in the light cone can enter the humaneye. Therefore, light rays emitted by display sub-regions covered by across section between the monitor and the light cone may be regarded aslight rays capable of entering the human eye, and these displaysub-regions are display sub-regions that affect imaging of the virtualimage point, namely, the first display sub-regions. Optionally, acoverage area of the virtual image point P on the monitor along avertical direction may be determined according to formulas (3) and (4),thereby determining the first display sub-region according to thecoverage area:

$\begin{matrix}{y_{\min} = {{- Y_{i\; 2}} = {\frac{\left( {v - d_{l}} \right)\left( {{0.5\; E} - y_{i}^{\prime}} \right)}{d_{v}} + y_{i}^{\prime} - \frac{\left( {v - d_{l}} \right)E}{d_{v}}}}} & (3) \\{y_{\max} = {{y_{i}^{\prime} + Y_{i\; 1}} = {\frac{\left( {v - d_{l}} \right)\left( {{0.5E} - y_{i}^{\prime}} \right)}{d_{v}} + y_{i}^{\prime}}}} & (4) \\{{wherein}\text{:}} & \; \\{\frac{Y}{E} = \frac{v - d_{l}}{d_{v}}} & (5) \\{\frac{Y_{i\; 1}}{{0.5\; E} - y_{i}^{\prime}} = \frac{v - d_{l}}{d_{v}}} & (6) \\{Y_{i\; 2} = {Y - Y_{i\; 1} - y_{i}^{\prime}}} & (7)\end{matrix}$

In the foregoing formulas: Y represents a vertical coverage area of across section, in a plane where the monitor is located, of a light conefrom the virtual image point P, which is used as a virtual light source,to the pupil of the human eye; Y_(i1) represents a vertical distancefrom an upper boundary of the cross section between the light cone andthe monitor to a plane 2, Y_(i2) represents a vertical distance from alower boundary of the cross section between the light cone and themonitor to a plane 1, wherein the plane 1 passes through the center ofthe monitor and is perpendicular to the plane of the monitor, and theplane 2 is a plane that passes through the virtual image point P and isparallel to the plane 1; y_(min) represents a minimum coordinate valueof a vertical coverage area of the virtual image point P on the monitor;y_(max) represents a maximum coordinate value of the vertical coveragearea of the virtual image point P on the monitor; d_(l) represents adistance between the monitor and the lenslet array; d_(v) represents adistance between the virtual image and the lenslet array; and Erepresents the diameter of the pupil.

(d2) According to a known start position of each display sub-region inthe vertical direction of the monitor, a serial number of a firstdisplay sub-region in the vertical coverage area of the virtual imagepoint P on the monitor is determined. Because a correspondingrelationship between sub-images and lenslets is known, a serial numberof a lenslet covered in the vertical direction can be obtained.

According to the coverage area and Y-coordinate position information ofeach display sub-region in the monitor, display sub-regionscorresponding to the coverage area can be determined. If the coveragearea completely overlaps with vertical position coordinates of thedisplay sub-regions, the display sub-regions at the correspondingvertical coordinates in the coverage area can be conveniently determinedas the first display sub-regions; if an edge part of the coverage areadoes not completely correspond to a complete display sub-region, thefirst display sub-region may be determined by using an approximateprocessing method, for example, a display sub-region partially coveredby the coverage area is determined as the first display sub-region, oronly a display sub-region completely covered by the coverage area is thefirst display sub-region, or a display sub-region whose coordinateoverlapping sub-area with the coverage area satisfies a particular ratiois the first display sub-region. The determining manner is veryflexible.

(e2) Likewise, by using a method similar to (c2) and (d2), a serialnumber of a first display sub-region in a horizontal-direction coveragearea of the virtual image point P on the monitor is determined, therebyobtaining a serial number of a first lenslet covered in the horizontaldirection.

Lenslets corresponding to the determined lenslet serial numbers are thefirst lenslets on which tilt control is going to be performed. In thissolution, the method for determining a lenslet that affects imagecapture in the depth distribution sub-region is simple and easy toimplement.

A person skilled in the art should understand that, in any of theforegoing methods in the specific implementation manners of the presentapplication, sequence numbers of the steps do not mean a correspondingexecution order. The corresponding execution order of the steps shouldbe determined according to functions and internal logic thereof, andshould not be construed as any limitation on implementation processes ofthe specific implementation manners of the present application.

FIG. 4 is a logic block diagram of a first light field display controlapparatus according to an embodiment of the present application. Asshown in FIG. 4, the light field display control apparatus provided inthe embodiment of the present application may comprise: a depthdistribution sub-region determining module 41 and a tilt control module42.

The depth distribution sub-region determining module 41 is configured todetermine a partial depth distribution sub-region of content accordingto at least depth distribution information of the content.

The tilt control module 42 is configured to tilt a first lenslet withrespect to an original plane of a lenslet array of a light field displaydevice according to at least a display DoF range of a light fielddisplay device and the depth distribution sub-region, wherein the firstlenslet is a lenslet that is in the lenslet array and affects display ofa first object, and the first object is a part, which is located in thedepth distribution sub-region, of the content.

In the technical solution provided by the embodiment of the presentapplication, the first lenslet is tilted with respect to the originalplane of the lenslet array according to at least a current display DoFrange of the light field display device and a depth distributionsub-region on which image quality control needs to be performed, tochange a display image distance of the first object in space at least bymeans of control over the tilt of the first lenslet, so that an averagedisplay image distance of a virtual image which is formed after thefirst object is displayed by the first lenslet is distributed in thedisplay DoF range of the light field display device as far as possible,thereby improving display quality of the first object displayed by thefirst lenslet. This solution can avoid complex computation on sourcecontent of a light field, and display quality of a same object can beadjusted by tilting a corresponding lenslet with respect to the originalplane of the lenslet array. The solution is simple and easy to control.

The light field display control apparatus may execute the light fielddisplay control method in application programs that comprise but notlimited to: image presentation, video playback, and the like, to performimage display control. A device manifestation form of the light fielddisplay control apparatus is not limited. For example, the light fielddisplay control apparatus may be an independent component, and thecomponent is in coordination and communication with a light fielddisplay device; or the light field display control apparatus may beintegrated in a light field display device as a functional module, andthe light field display device may comprise but not limited to: anelectronic device having a light field display capability. For example,the light field display device may comprise but not limited to: anear-to-eye light field display device, and the near-to-eye light fielddisplay device may comprise but not limited to: a smart helmet, smartglasses, and the like.

Optionally, an average circle of confusion, in a first display plane ofthe light field display device, of the first object displayed by thetilted first lenslet is less than or equal to a permissible circle ofconfusion, wherein the first display plane is a display plane that iscorresponding to a focusing distance for viewing in a depth directionand perpendicular to the depth direction. In this solution, reduction ofthe average circle of confusion in the first display plane can be usedas a convergence condition for tilt control on the first lenslet,thereby improving the efficiency and pertinence of the tilt control onthe first lenslet.

With reference to any technical solution provided by the embodiments ofthe present application, as shown in FIG. 5, optionally, the depthdistribution sub-region determining module 41 comprises: an informationacquiring sub-module 411 and a first depth distribution sub-regiondetermining sub-module 412. The information acquiring sub-module 411 isconfigured to acquire ROI information and the depth distributioninformation; and the first depth distribution sub-region determiningsub-module 412 is configured to determine the depth distributionsub-region according to the ROI information and the depth distributioninformation. In this solution, the depth distribution information of thecontent and the interest information are combined to determine a partialdepth distribution sub-region of the content as the depth distributionsub-region on which display quality control needs to be performed, sothat the determining of the depth distribution sub-region fits an actualrequirement of the user more desirably, and a personalized applicationrequirement of the user can be better satisfied.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, the depth distribution sub-regiondetermining module 41 comprises: an image analysis sub-module 413 and asecond depth distribution sub-region determining sub-module 414. Theimage analysis sub-module 413 is configured to perform image analysis onthe content; and the second depth distribution sub-region determiningsub-module 414 is configured to determine the depth distributionsub-region according to a result of the image analysis and the depthdistribution information. In this solution, the result of the imageanalysis on the content and the depth distribution information of thecontent are combined to automatically determine the depth distributionsub-region on which display quality control needs to be performed, sothat the determining of the depth distribution sub-region is moreintelligent, and the efficiency and universality of determining thedepth distribution sub-region are improved.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, as shown in FIG. 6, the tiltcontrol module 42 comprises: an expected display depth informationacquiring sub-module 421, an expected tilt angle determining sub-module422, and a first tilt control sub-module 423. The expected display depthinformation acquiring sub-module 421 is configured to determine expecteddisplay depth information of the first object according to the displayDoF range and the depth distribution sub-region; the expected tilt angledetermining sub-module 422 is configured to determine an expected tiltangle of the first lenslet according to at least a focal length of thefirst lenslet and the expected display depth information; the first tiltcontrol sub-module 423 is configured to tilt the first lenslet withrespect to the original plane of the lenslet array according to at leastthe expected tilt angle. This solution improves the tilt controlefficiency of the first lenslet, and the first object displayed by thetilted first lenslet can obtain a relatively large clear imaging area.

Further optionally, the first tilt control sub-module 423 comprises: afirst tilt direction determining unit 4231 and a first tilt control unit4232. The first tilt direction determining unit 4231 is configured todetermine a tilt direction of the first lenslet according to a relativedistribution of the depth distribution sub-region and the display DoFrange in the depth direction; and the first tilt control unit 4232 isconfigured to tilt the first lenslet with respect to the original planeof the lenslet array according to the tilt direction and the expectedtilt angle. By means of this solution, the tilt control efficiency ofthe first lenslet can be improved, so that tilt control on the firstlenslet satisfies the convergence condition of the average circle ofconfusion as quickly as possible.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, the tilt control module 42comprises: an expected display depth information acquiring sub-module421, a permissible tilt angle range determining sub-module 424, and asecond tilt control sub-module 425. The expected display depthinformation acquiring sub-module 421 is configured to determine expecteddisplay depth information of the first object according to the displayDoF range and the depth distribution sub-region; the permissible tiltangle range determining sub-module 424 is configured to determine apermissible tilt angle range of the first lenslet according to at leastthe expected display depth information; and the second tilt controlsub-module 425 is configured to tilt the first lenslet with respect tothe original plane of the lenslet array according to at least thepermissible tilt angle range. This solution improves the tilt controlefficiency of the first lenslet, and can improve the clearness of thefirst object displayed by the tilted first lenslet.

Further optionally, the permissible tilt angle range determiningsub-module 424 comprises: an expected tilt angle determining unit 4241and a permissible angle range determining unit 4242. The expected tiltangle determining unit 4241 is configured to determine an expected tiltangle of the first lenslet according to at least a focal length of thefirst lenslet and the expected display depth information; and thepermissible angle range determining unit 4242 is configured to determinethe permissible angle range according to at least the expected tiltangle. The permissible angle range determined by using this solution isrelatively reasonable, and control over the tilt of the lenslet withreference to the imaging plane performed based on the permissible anglerange is efficient and easy to implement.

Further optionally, the second tilt control sub-module 425 comprises: asecond tilt direction determining unit 4251 and a second tilt controlunit 4252. The second tilt direction determining unit 4251 is configuredto determine a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and the second tilt controlunit 4252 is configured to tilt the first lenslet with respect to theoriginal plane of the lenslet array according to the tilt direction andthe permissible tilt angle range. By means of this solution, the tiltcontrol efficiency of the first lenslet can be improved, so that tiltcontrol on the first lenslet satisfies the convergence condition of theaverage circle of confusion as quickly as possible.

Optionally, as shown in FIG. 7, the expected display depth informationacquiring sub-module 421 comprises: a first expected display depthinformation acquiring sub-module 4211. The first expected display depthinformation acquiring sub-module 4211 is configured to determine anydisplay depth in the display DoF range as the expected display depthinformation of the first object. This solution of determining theexpected display depth information is simple, and the implementationmanner is flexible.

Optionally, the expected display depth information acquiring sub-module421 comprises: a second expected display depth information acquiringsub-module 4212. The second expected display depth information acquiringsub-module 4212 is configured to determine a display depth, which is inthe display DoF range and close to the depth distribution sub-region, asthe expected display depth information of the first object according toa relative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction. By means of this solution, theexpected display depth information is determined and subsequent displaycontrol is performed according to the expected display depthinformation, so that the sense of depth of the user during watching isnot reduced while display clearness is improved, thereby improving userexperience.

Optionally, as shown in FIG. 8, the light field capture controlapparatus further comprises: a lenslet determining module 43. Thelenslet determining module 43 is configured to determine a lenslet,which is in the display array and affects display of the first object,as the first lenslet. After the first lenslet to be adjusted isdetermined, tilt control may be performed on the first lenslet toimprove display quality of the first object.

Further optionally, the lenslet determining module 43 comprises: adisplay sub-region determining sub-module 431 and a lenslet determiningsub-module 432. The display sub-region determining sub-module 431 isconfigured to determine a first display sub-region according to a lightfield image corresponding to the content, wherein the first displaysub-region is a display sub-region that is in the monitor of the lightfield display device and affects display of the first object; and thelenslet determining sub-module 432 is configured to determine, accordingto a mapping relationship between display sub-regions in the monitor andlenslets in the lenslet array, a lenslet which is in the lenslet arrayand corresponding to the first display sub-region as the first lenslet.This solution of determining the first lenslet is simple and easy toimplement.

FIG. 9 is a structural block diagram of a light field display controlapparatus according to an embodiment of the present application. Thespecific embodiment of the present application does not limit a specificimplementation manner of the light field display control apparatus 900.As shown in FIG. 9, the light field display control apparatus 900 maycomprise:

a processor 910, a communications interface 920, a memory 930, and acommunications bus 940.

The processor 910, the communications interface 920 and the memory 930communicate with each other by using the communications bus 940.

The communications interface 920 is configured to communicate with adevice having a communications function, an external light source, andthe like.

The processor 910 is configured to execute a program 932, andspecifically may execute related steps in any one of the foregoingembodiments of the light field display control method.

For example, the program 932 may comprise program code, and the programcode comprises a computer operation instruction.

The processor 910 may be a central processing unit (CPU), or anapplication specific integrated circuit (ASIC), or is configured to beone or more integrated circuits for implementing the presentapplication.

The memory 930 is configured to store the program 932. The memory 930may comprise a random access memory (RAM), or may further comprise anon-volatile memory, for example, at least one magnetic disk memory.

For example, in an optional implementation manner, the processor 910 mayexecute the following steps by executing the program 932: determining apartial depth distribution sub-region of content according to at leastdepth distribution information of the content; and tilting a firstlenslet at least with respect to an original plane of a lenslet array ofa light field display device according to a display DoF range of thelight field display device and the depth distribution sub-region,wherein the first lenslet is a lenslet that is in the display array andaffects display of a first object, and the first object is a part, whichis located in the depth distribution sub-region, of the content.

In another optional implementation manner, the processor 910 may furtherexecute steps mentioned in any other embodiment above by executing theprogram 932, and details are not described herein again.

For specific implementation of the steps in the program 932, refer tocorresponding descriptions in the corresponding steps, modules,sub-modules, and units in the foregoing embodiments, and details are notdescribed herein again. It may be clearly understood by a person skilledin the art that, for the purpose of convenient and brief description,for a specific working process of the foregoing system, apparatus, andunit, reference may be made to a corresponding process in the foregoingmethod embodiments, and details are not described herein again.

An embodiment of the present application further provides a light fielddisplay device. As shown in FIG. 10, the light field display devicecomprises: a monitor, a lenslet array, and a light field display controlapparatus; the monitor and the lenslet array are arranged close to eachother (that is, a distance between the monitor and the lenslet array isvery small, and is usually less than a maximum focal length of thelenslets in the lenslet array); the lenslet array comprises multipletiltable lenslets distributed in the array, and the light field displaycontrol apparatus is connected to the monitor and the lenslet array, andperforms tilt control on one or more lenslets to improve partial displayclearness. The light field display control apparatus may be any lightfield display control apparatus provided in the embodiments of thepresent application, and can execute, in application processes thatcomprise but not limited to: image presentation, video playback, and thelike, any light field display control method provided in the embodimentsof the present application, to perform image display control. For aprinciple and an apparatus structure for the light field display controlperformed by the light field display device, refer to the description inother embodiments of the present application, and details are notdescribed herein again.

In the embodiments of the present application, the serial numbers and/orcorresponding order of the embodiments is merely used for description,and is not intended to represent merits of the embodiments. Thedescriptions on the embodiments have their respective emphasis, and fora part that is not described in detail in a certain embodiment,reference may be made to related description in another embodiment. Forrelated descriptions of implementation principles or processes relatedto the apparatus, device or system embodiments, reference may be made tothe records of the corresponding method embodiments, which are notrepeated herein.

A person of ordinary skill in the art should appreciate that, incombination with the examples described in the embodiments herein, unitsand method steps can be implemented by electronic hardware, or acombination of computer software and electronic hardware. Whether thefunctions are executed by hardware or software depends on specificapplications and design constraint conditions of the technicalsolutions. A person skilled in the art can use different methods toimplement the described functions for every specific application, but itshould not be considered that this implementation goes beyond the scopeof the present application.

When being implemented in the form of a software functional unit andsold or used as a separate product, the functions may be stored in acomputer-readable storage medium. Based on such understanding, thetechnical solutions of the present invention essentially, or the partcontributing to the prior art, or part of the technical solutions may beimplemented in a form of a software product. The computer softwareproduct is stored in a storage medium, and comprises severalinstructions for instructing a computer device (which may be a personalcomputer, a server, a network device, or the like) to execute all orpart of the steps of the method described in each of the embodiments ofthe present application. The aforementioned storage medium comprises:any medium that can store program codes, such as a USB disk, a removablehard disk, a read-only memory (ROM), a random access memory (RAM), amagnetic disk, or an optical disc.

In the embodiments of the apparatus, method and system of the presentapplication, apparently, each component (such as the system, sub-system,module, sub-module, unit, and sub-unit) or each step may be decomposed,combined and/or combined after being decomposed. The decompositionand/or re-combination should be considered as an equivalent solution ofthe present application. Meanwhile, in the description of the specificembodiments of the present application, a feature described and/or shownfor one implementation manner may be used in one or more otherimplementation manners in an identical or similar way, may be combinedwith a feature in another implementation manner, or may replace afeature in another implementation manner.

It should be emphasized that the term “comprise/contain” used in thistext to indicate existence of a feature, element, step or component, anddoes not exclude existence or addition of one or more other features,elements, steps or components.

Finally, it should be noted that: the aforementioned description of theimplementation manners are merely provided for describing the presentapplication, but not intended to limit the present application. A personof ordinary skill in the art can also make many variations and changeswithout departing from the spirit and the scope of the presentapplication. Therefore, all the equivalent technical solutions also fallwithin the scope of the present application, and the patent protectionscope of the present application shall be limited by the claims.

What is claimed is:
 1. A light field display control method, comprising:determining a depth distribution sub-region of content according to atleast depth distribution information of the content; and tilting a firstlenslet with respect to an original plane of a lenslet array of a lightfield display device according to at least a display depth of field(DoF) range of the light field display device and the depth distributionsub-region, wherein the first lenslet is a lenslet that is in thelenslet array and affects display of a first object, and the firstobject is a part, which is located in the depth distribution sub-region,of the content; wherein tilting the first lenslet with respect to theoriginal plane of the lenslet array of the light field display deviceaccording to at least the display DoF range of the light field displaydevice and the depth distribution sub-region comprises: determiningexpected display depth information of the first object according to thedisplay DoF range and the depth distribution sub-region; determining anexpected tilt angle or a permissible tilt angle range of the firstlenslet according to at least a focal length of the first lenslet andthe expected display depth information; and tilting the first lensletwith respect to the original plane of the lenslet array according to atleast the expected tilt angle or the permissible tilt angle range. 2.The method of claim 1, wherein an average circle of confusion, in afirst display plane of the light field display device, of the firstobject displayed by the tilted first lenslet is less than or equal to apermissible circle of confusion, and the first display plane is adisplay plane that is corresponding to a focusing distance for viewingin a depth direction and perpendicular to the depth direction.
 3. Themethod of claim 1, wherein the determining a depth distributionsub-region of content according to at least depth distributioninformation of the content comprises: acquiring region of interest (ROI)information and the depth distribution information; and determining thedepth distribution sub-region according to the ROI information and thedepth distribution information.
 4. The method of claim 1, wherein thedetermining a depth distribution sub-region of content according to atleast depth distribution information of the content comprises:performing image analysis on the content; and determining the depthdistribution sub-region according to a result of the image analysis andthe depth distribution information.
 5. The method of claim 1, whereinthe tilting the first lenslet with respect to the original plane of thelenslet array according to at least the expected tilt angle comprises:determining a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and tilting the first lensletwith respect to the original plane of the lenslet array according to thetilt direction and the expected tilt angle.
 6. The method of claim 1,wherein the determining a permissible tilt angle range of the firstlenslet according to at least a focal length of the first lenslet andthe expected display depth information comprises: determining anexpected tilt angle of the first lenslet according to at least the focallength of the first lenslet and the expected display depth information;and determining the permissible angle range according to at least theexpected tilt angle.
 7. The method of claim 1, wherein the tilting thefirst lenslet with respect to the original plane of the lenslet arrayaccording to at least the permissible tilt angle range comprises:determining a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and tilting the first lensletwith respect to the original plane of the lenslet array according to thetilt direction and the permissible tilt angle range.
 8. The method ofclaim 1, wherein the determining expected display depth information ofthe first object according to at least the display DoF range comprises:determining any display depth in the display DoF range as the expecteddisplay depth information of the first object.
 9. The method of claim 1,wherein the determining expected display depth information of the firstobject according to at least the display DoF range comprises:determining a display depth, which is in the display DoF range and closeto the depth distribution sub-region, as the expected display depthinformation of the first object according to a relative distribution ofthe depth distribution sub-region and the display DoF range in the depthdirection.
 10. The method of claim 1, wherein before the tilting a firstlenslet with respect to an original plane of a lenslet array of a lightfield display device, the method further comprises: determining alenslet which is in the lenslet array and affects display of the firstobject as the first lenslet.
 11. The method of claim 10, wherein thedetermining a lenslet which is in the lenslet array and affects displayof the first object as the first lenslet comprises: determining a firstdisplay sub-region according to the expected display depth information,the focusing distance for viewing, and a pupil size of a human eye,wherein the first display sub-region is a display sub-region that is inthe monitor of the light field display device and affects display of thefirst object; and determining, according to a mapping relationshipbetween display sub-regions in the monitor and lenslets in the lensletarray, a lenslet which is in the lenslet array and corresponding to thefirst display sub-region as the first lenslet.
 12. A light field displaycontrol apparatus, comprising: a processor, a communications interface,a memory, and a communications bus, wherein the processor, thecommunications interface, and the memory communicate with each other bymeans of the communications bus; wherein the memory is configured tostore at least one instruction, and the instruction causes the processorto perform operations including: determining a depth distributionsub-region of content according to at least depth distributioninformation of the content; and tilting a first lenslet at least withrespect to an original plane of a lenslet array of a light field displaydevice according to a display depth of field (DoF) range of the lightfield display device and the depth distribution sub-region, wherein thefirst lenslet is a lenslet that is in the lenslet array and affectsdisplay of a first object, and the first object is a part, which islocated in the depth distribution sub-region, of the content; whereintilting the first lenslet with respect to the original plane of thelenslet array of the light field display device according to at leastthe display DoF range of the light field display device and the depthdistribution sub-region comprises: determining expected display depthinformation of the first object according to the display DoF range andthe depth distribution sub-region; determining an expected tilt angle ora permissible tilt angle range of the first lenslet according to atleast a focal length of the first lenslet and the expected display depthinformation; and tilting the first lenslet with respect to the originalplane of the lenslet array according to at least the expected tilt angleor the permissible tilt angle range.
 13. A light field display device,comprising: a monitor; a lenslet array, wherein the lenslet arraycomprises multiple tiltable lenslets distributed in the array; and alight field display control apparatus of claim 12, wherein the lightfield display control apparatus is connected to the monitor and thelenslet array.
 14. The apparatus of claim 12, wherein an average circleof confusion, in a first display plane of the light field displaydevice, of the first object displayed by the tilted first lenslet isless than or equal to a permissible circle of confusion, and the firstdisplay plane is a display plane that is corresponding to a focusingdistance for viewing in a depth direction and perpendicular to the depthdirection.
 15. The apparatus of claim 12, wherein determining a depthdistribution sub-region of content according to at least depthdistribution information of the content comprises: acquiring region ofinterest (ROI) information and the depth distribution information; anddetermining the depth distribution sub-region according to the ROIinformation and the depth distribution information.
 16. The apparatus ofclaim 12, wherein the determining a depth distribution sub-region ofcontent according to at least depth distribution information of thecontent comprises: performing image analysis on the content; anddetermining the depth distribution sub-region according to a result ofthe image analysis and the depth distribution information.
 17. Theapparatus of claim 12, wherein the tilting the first lenslet withrespect to the original plane of the lenslet array according to at leastthe expected tilt angle comprises: determining a tilt direction of thefirst lenslet according to a relative distribution of the depthdistribution sub-region and the display DoF range in the depthdirection; and tilting the first lenslet with respect to the originalplane of the lenslet array according to the tilt direction and theexpected tilt angle.
 18. The apparatus of claim 12, wherein thedetermining a permissible tilt angle range of the first lensletaccording to at least a focal length of the first lenslet and theexpected display depth information comprises: determining an expectedtilt angle of the first lenslet according to at least the focal lengthof the first lenslet and the expected display depth information; anddetermining the permissible angle range according to at least theexpected tilt angle.
 19. The apparatus of claim 12, wherein the tiltingthe first lenslet with respect to the original plane of the lensletarray according to at least the permissible tilt angle range comprises:determining a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and tilting the first lensletwith respect to the original plane of the lenslet array according to thetilt direction and the permissible tilt angle range.
 20. The apparatusof claim 12, wherein the determining expected display depth informationof the first object according to at least the display DoF rangecomprises: determining any display depth in the display DoF range as theexpected display depth information of the first object.
 21. Theapparatus of claim 12, wherein the determining expected display depthinformation of the first object according to at least the display DoFrange comprises: determining a display depth, which is in the displayDoF range and close to the depth distribution sub-region, as theexpected display depth information of the first object according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction.
 22. The apparatus of claim 12,wherein before the tilting a first lenslet with respect to an originalplane of a lenslet array of a light field display device, the operationsfurther comprises: determining a lenslet which is in the lenslet arrayand affects display of the first object as the first lenslet.
 23. Theapparatus of claim 22, wherein the determining a lenslet which is in thelenslet array and affects display of the first object as the firstlenslet comprises: determining a first display sub-region according tothe expected display depth information, the focusing distance forviewing, and a pupil size of a human eye, wherein the first displaysub-region is a display sub-region that is in the monitor of the lightfield display device and affects display of the first object; anddetermining, according to a mapping relationship between displaysub-regions in the monitor and lenslets in the lenslet array, a lensletwhich is in the lenslet array and corresponding to the first displaysub-region as the first lenslet.